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Remarks / Arguments 

Status of the Claims 

Claims 1-20 were pending in the application. After the entry of the amendments 
herein, claims 1, 2-5, 8, 11-17, 19-24, 27, 30-35, and 37-55 are pending. Claims 1, 2, 4- 
5, 8, 1 1-1 7, 1 9-24, 27, 30-35 and 37 have been amended by the present amendment. 
Claims 38-55 have been added- Claims 6, 7, 9, 10, 18, 25, 26, 28, 29, and 36 have been 
cancelled. 

Upon entry of this Amendment and Response, there will be a total of 45 pending 
claims, with two independent claims, claims 1 and 20. Applicant originally filed and paid 
for 37 total claims, with 2 independent claims, and therefore the additional fee is 
enclosed. 

Summary of Support For Claim Amendments 

Support for amended claims 1 and 20 can be found in the specification at least at 
page 16 line 1 8 and page 17 line 8. Support for amended claims 8 and 27 can be found at 
least at page 17 line 8. Support for amended claims 1 1 and 30 can be found at least at 
page 1 1 line 26. Support for amended claims 12 and 31 can be found at least at page 17 
line 1 . Support for amended claims 1 5 and 34 can be found at least at page 17 line 1 and 
page 1 8 line 26. Support for amended claims 17 and 25 can be fount at least at page 14 
line 28. Support for amended claims 19 and 37 can be found at least at page 17 line 10. 
Additional support for the claim amendments can be found in the claims as originally 
filed. Support for new claims 38 and 47 can be found at least at page 1 5 line 1 1 . Support 
for new claims 39, 40, 48, and 49 can be found at least at originally filed claim 9. 
Support for new claims 41, 42, 50 and 51 can be found at least at page 19 line 7. Support 
for new claims 43 and 52 can be found at least at originally filed claims 7 and 1 8, 
Support for new claims 44 and 53 can be found at least at page 15 line 1 8. Support for 
new claims 45 and 54 can be found at least at page 18 line 9. Support for new claims 46 
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and 55 can be found at least at originally filed claim 16. The remainder of the claim 
amendments were made to better clarify the invention- No new matter has been added. 

Claim Rejections 

Claims 1-6 and 20-25 are rejected under 35 U.S.C. § 103(a) as being unpatentable 
over U.S. Patent No. 6,282,549 ("Hoeffert") in view of U.S. Patent No. 5,729,741 
("Liaguno"). Claims 7-10, 12-19, 26-29, and 31-37 are rejected under 35 U.S.C. § 103(a) 
as being unpatentable over Hoeffert in view of Liaguno and further in view of U.S. Patent 
No. 5,655,1 17 ("Goldberg"). Claims 1 1 and 30 are rejected under 35 U.S.C. § 103(a) as 
being unpatentable over Hoeffert in view of Liaguno, further in view of Goldberg and 
further in view of U.S. Patent No. 6,026,41 1 ("Delp"). 

Cited References 

Hoeffert 

Hoeffert is directed to a method and apparatus for "searching the Internet" and for 
providing "analysis of the content of files found in the search" (col. 2, lines 22-25). 
Hoeffert describes "scan[ing] the media description file in each directory at a web site, 
and add[ing] the text based information stored there into the index being created by the 
crawler." (col. 3, line 66 through coL 4 line 2). Examples of the text used to index the 
media file includes the URL, text strings, titles, keywords, and annotations, (see col. 6 
lines 15-32). Hoeffert also describes "searching based on information signals stored 
inside the content 5 5 (col. 8 lines 39-40). Such indices can include "a motion metric and 
brightness, contrast and color estimate." (col. 8 lines 42-43). 
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Ljagyno 

Liaguno is directed to a system for storage and retrieval of information from a 
computer employing a "user-defined storage index and a supplemental text description- 
based format" (col. 2, lines 37-38). Liaguno describes creating a user-created searchable 
index in connection with importing media images into a computer system via a keyword 
or profile index for the associated coupled media and text files (see abstract; col. 4, lines 
38-54). The user may then use a text file to identify a media image* "all media inputs into 
the system. . .are processed to create a 6 text' file for that media image, so that a free text 
search operator may be employed to locate a media image file, and thereby access the 
index file with which the text file is associated" (col. 8, lines 44-49). In the disclosed 
system, "all media inputs to the system are processed by means of a text-generator in 
order to create a *text' file for that media image" (col. 3, lines 32-34). For non-text 
media, various other methods are used to "create a text file for that image so that a free 
text search operator may be employed to locate a media image file." (col. 8, lines 46-50). 
For example, audio files may be processed using a "commercially available voice 
recognition-to-text translation routine" (col. 9 32-53), whereas text in photographs "can 
be detected by processing the image through a character recognition routine," (col. 10, 
lines 10-1 1). However, "[i]f the original image contains no text of any kind, then the text 
image will also contain not text, and will be identified as being 'blank text'." (col. 10, 
lines 42-44). 

Goldberg 

Goldberg is directed to a system and method for indexing a multimedia data 
stream (See Abstract), To create the index filed described by Goldberg, the user 
"identifies [an] object either by drawing a region around the desired object or by usiflg 
more sophisticated image processing techniques to. differentiate the pixels of a particular 
object from the pixels of other objects or the background image." (col. 4, lines 1 5-19). 
Once the object is identified, "a set of information pertaining to the object" is created 
which "provides a means for inputting additional information which further describes the 
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target." (col. 4 lines 20-24). In summary, the system and method described my Goldberg 
allows a user to manually select an object in a video frame, and "performing] video 
indexing" by entering relevant information that can subsequently be searched, (col. 4 3 
line 29). 

Delp 

Delp is directed to methods and systems for "building an image index and for 
querying by image colors images from the internet" (col. 1, lines 45-47). Delp describes 
creating an index file by "extracting] color information from several levels within an 
image and mak[ing] it available for searching.' 1 (col, 4, lines 27-28). In one embodiment 
where "efficiency is not a concern," (col. 3 line 34) Delp describes a creating a histogram 
to "select the most popular samples" (col. 3 line 36) by "generat[ing] a variable size for 
the histogram bins depending on precision desired" (col. 5 lines 30-32) and determining 
the most popular colors based on the number of pixels that fell into each bin. (FIG. 5). 
The resulting color samples are then used to create an index for the image, (col. 3 lines 
40-41). 

Independent Claims 1 and 20 

Claims 1-6 and 20-25 are rejected under 35 U.S.C § 103(a) as being unpatentable 
over Hoeffert in view of Liaguno. 

Neither Hoeffert nor Liaguno, either alone or in combination, teach or suggest 
generating a data string for use in indexing a media element based on relationships among 
a subset of components within a media file. With regard to media files generally, . 
Hoeffert teaches creating a "media index [that] is generated by storing . . + information in 
an index format." (col. 7 lines 20-21). The information used to create the media index 
includes "relevant lexical information" such a$: 

(a) "the name of the media file 

(b) URL of the media file 

(c) text string which is associated with the media file anchor reference 
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(d) title of the HTML document containing the media file 

(e) keywords associated with the HTML document 

(f) URL for the HTML document containing the media file reference 

(g) keywords embedded in the media file 

(h) textual annotations in the media file 

(i) script dialogue, closed captioning and lyric data in the media file 
(j) auxiliary data in the media file (copyright, author, producer, etc.) 

(k) auxiliary data located within die media reference in the HTML document 
and 

(1) auxiliary data located in an associated media description file" 

(col. 6 lines 14-32). The textural information is then used to create a "stored a media 
description file. . . [containing] a series of records of textual information for each media 
file within the current directory" (col. 3, lines 60-65). For example, the method taught by 
Hoefifert relies on author-supplied textual information such as HTML tags, surrounding 
text, or a URL (see col. 4, lines 24-45). 

Hoefifert also describes a "scheme for determining the average frame difference 
for a pixel in a sequence of video" (col. 9, lines 37-39). To make such a determination, 
Hoeffert describes using attributes of a video file "to determine if a given video file 
contains low 9 medium, or high amounts of motion" by calculating "a singled valued 
scalar" referred to as a "motion metric" (col. 9, lines 25-29). The motion metric, as 
described by Hoeffert, represents the M visual change activity" (col. 10 line 7) and is 
calculated by sampling some number of video frames and computing the differences in 
RGB values between each pixel for each frame. (See col. 10, lines 23-43). Similar 
methods are described with reference to motion vectors. (See col. 12, lines 1-18). 
However, Hoeffert does not teach or suggest determining a component value, selecting a 
subset of components have a substantially similar value, and using the relationships 
among the subset of components to create a data string for use in indexing a media 
element, as recited and amended in Applicant's independent claims 1 and 20. In contrast, 
Hoeffert analyses differences between the same pixel from one frame to another to 
determine the degree of change in that pixel. 

Liaguno does not cure the deficiencies of Hoeffert. As described above, Liaguno 
uses various forms of text recognition to build text-based index files for media elements, 
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but does not select a subset of components of the media file based on a component value 
and use the relationships among the components to create a data string for indexing the 
media element 

Accordingly, Applicant respectfully submits that independent claims 1 and 20 are 
patentable over Hoeffert, either alone or in combination with Liaguno. Applicant further 
submits that claims 2-5, 8 a 11-17, 19,21-24,27, 30-35, and 37-55, which depend from 
independent claims 1 and 20 respectively, incorporate all of the limitations therein, and 
therefore are also patentable over each of the cited references, either alone or in 
combination. Applicants respectfully request reconsideration and withdrawal of the 
rejections of claims 1-5, 8, 1 1-17, 19-24, 27, 30-35, and 37-55. 

Claims 7-10 and 12-19 

Claims 7-10 and 12-19 are rejected under 35 U«S,C. § 103(a) as being 
unpatentable over Hoeffert in view of Liaguno and further in view of Goldberg. Claims 
7, 9, 10, and 18 have been cancelled by this Amendment and Response, and renders the 
rejections of those claims moot 

Applicant believes Goldberg teaches a method of manually selecting and indexing 
media elements throughout a video stream, and therefore Applicants respectfully suggest 
that Goldberg does not cure the deficiency of Hoeffert in view of Liaguno with respect to 
claims 8 7 12-17, and 19. 

Goldberg describes a user selection process whereby a user identifies a portion of 
a media element (e.g., by drawing a box around the portion of the image that is of interest 
to the user) and subsequently allows the user to create an index for the image. Goldberg 
does not teach or suggest using the attributes of a media element (e.g., component values) 
to create a data string to be used for indexing the element. In contrast, Applicant's 
method discloses determining a subset of components within a media element that share a 
common pixel value and using the relationships among the subset of pixels to create a 
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data string. Therefore, Applicants invention is patentable over Hoeffert in view of 
Liaguno, further in view of Goldberg. 

Applicants respectfully submit that even if the references provided motivation for 
their combination, which they do not, claims 7-10 and 12-19 1 1 are patentable over 
Hoeffert, alone or in combination with Liaguno, and Goldberg, and request 
reconsideration and withdrawal of the rejection of claims 8, 12-17, and 19 under 35 
U.S.C. §103(a). 

Claims 11 and 30 

Claims 1 1 and 30 are rejected under 35 U.S.C. § 103(a) as being unpatentable 
over Hoeffert in view of Liaguno, further in view of Goldberg, and further in view of 
Delp. Generally, Applicant believes that Delp teaches comparing an index file generated 
based on the distribution of color within the image to an inventory of images or indices. 

Applicant respectfully submits that Delp does not cure the deficiencies of 
Hoeffert, Liaguno and Goldberg. Delp does not teach or suggest determining a 
component value, selecting a subset of components have a substantially similar value, and 
using the relationships among the subset of components to create a data string for use in 
indexing a media element, as recited in Applicant's amended independent claims 1 and 
20 from which claims 1 1 and 30 depend, respectively. Nor does Delp teach or suggest 
genera ting a histogram band for each of a plurality of component values for the subset of 
components, as recited in Applicant's new claims 38 and 47, from which claims 1 1 and 
30 depend. 

Applicants respectfully submit that even if the references provided motivation for 
their combination, which they do not, claims 1 1 and 30 are patentable over Hoeffert, 
Liaguno, Goldberg, and Delp, either alone or in combination, and request reconsideration 
and withdrawal of the rejection of claims 1 1 and 30 under 35 U.S.C. § 103(a). 
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Conclusion 



Applicant respectfully submits that Hoeffert, Liaguno, Goldberg, and Delp, either 
alone or in any combination, fail to teach each and every element of Applicant's 
independent claims 1 or 20, and those claims that depend directly or indirectly from those 
independent claims. 

Applicant respectfully requests reconsideration, withdrawal of all grounds of 
rejection, and allowance of all pending claims in due course. The Examiner is invited to 
contact Applicant's undersigned representative by telephone at the number listed below to 



discuss any outstanding issues. 



Respectfully submitted, 



Date: January ,2005 

Testa, Hurwitz & Thibeault, LLP 
High Street Tower 
125 High Street 
Boston, MA 02110 
(617) 248-7738 



Thomas A. Turano 
Attorney for Applicant 
Reg. No. 35,722 



3149567 
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